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Abstract 



In this paper, we use data drawn from the National Educational Longitudinal Study of 1988, which 
allows students to be linked to particular teachers and classes, to estimate the impact of observable 
and unobservable schooling characteristics on student outcomes. A variety of models show some 
schooling resources (in particular teacher qualifications) to be significant in influencing tenth grade 
mathematics test scores. Unobservable school, teacher, and class characteristics are important in 
explaining student achievement but do not appear to be correlated with observable variables in our 
sample. Thus, our results suggest that the omission of unobservables does not cause biased estimates 
in standard educational production functions. 
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L Introduction 



Over the past twenty five years the United States has significantly increased real per pupil 
expenditures on K-12 schooling. Despite this there is a widespread perception, based on unflattering 
international comparisons and time trends in standardized test scores, that the nation's public schools 
are failing. 1 The past decade, spurred by a spate of reports in the early 1980s, has seen numerous 
reforms scattered throughout the country, which have included new forms of student assessment, 
curriculum innovation and new classroom organization, as well as changes in school governance and 
financing. These reforms have been accompanied by increased research by economists, sociologists 
and others seeking to investigate the relationship between educational productivity and schooling 
expenditures. This literature, dating back to the 1966 Coleman report (Coleman et al., 1966), 
consists of literally hundreds of studies. Most have modelled standardized test scores across 
students, schools, or school districts, as a function of individual and family background characteristics 
and schooling variables such as expenditures per pupil and class size. This work concludes that 
individual and family background traits explain the vast majority of variation in student test scores. 
The effects of educational inputs such as per pupil spending, teacher experience, and teacher degree 
level have been shown to be relatively unimportant predictors of outcomes, and the impact of any 
particular input to be inconsistent across studies (Hanushek, 1986). 

The results with regard to teachers are particularly puzzling. Few would deny that teachers 
play a central role in the education of our nation's children. Teaching is the largest profession in the 
United States, employing over three million adults (NCES, 1994, p. 71). An elaborate system of 
teacher education and certification is geared towards the preparation of those entering teaching, and 
there are significant professional development opportunities for those remaining in the profession. 
More than 40% of teachers have at least a Master degree and more than 25% have at least twenty 
years full-time teaching experience (NCES, 1994, p. 77). Over 60% of all schooling expenditures 
at the K-12 level are devoted to instructional costs which consist overwhelmingly of teacher salaries 
and benefits. Further, teacher salary incentives reward years of experience and degree levels, traits 
that do not appear to have a relationship to student achievement. What can explain the inconsistent 



1 This view is not unchallenged. See for example Berliner and Biddle (1995). 
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findings of the educational productivity literature with respect to educational resources, particularly 
teachers? In this paper we shed some light on this question by systematically exploring the 
relationship between student achievement and schooling inputs. 

We utilize student level data from the National Educational Longitudinal Study of 1988 
(NELS), one of the most comprehensive nationally representative datasets available for educational 
productivity analyses. These data contain information on the background of 8th grade students in 
1988, who were re-surveyed in 1990 and 1992. Unlike most other data, NELS links students to a 
specific class and teacher, and thus marks a significant improvement over most previous surveys. 
We start with a traditional model at the individual student level, modelling mathematics achievement 
as a function of individual and family background characteristics for our sample of public school 
students. We add a familiar set of schooling, teacher and classroom characteristics to our model. 
Following Murnane and Phillips (1981) we broaden the measured teacher characteristics by also 
focusing on a set of variables describing teacher behavior. 

We then seek to determine the effect of omitting unobservable school or teacher or classroom 
traits on the estimated relationship between observable schooling characteristics and achievement. 
We estimate a variety of econometric models including those with fixed and random teacher effects, 
and auxiliary regressions in which consistent estimates of the effects of observable teacher 
characteristics on test scores are obtained by regressing estimated teacher fixed effects on these 
characteristics. We find that several teacher characteristics (in particular, teachers' math specific 
preparation) do appear to matter-that is, they are significant and influence student achievement in 
the expected direction. Further, there is little evidence that unobservable schooling characteristics 
omitted from standard educational production function models lead to systematic bias in the estimates 
of the coefficients of observable schooling variables. 

II. Background: Previous Literature on Educational Productivity 

In this section we briefly discuss the "educational productivity" literature. Over the past 
twenty five years a set of predominantly empirical studies conducted by economists have attempted 



to establish the relationship between schooling resources and student outcomes (Hanushek, 1986). 2 
Outcomes have usually been measured by standardized test scores, although in recent years there has 
been a resurgence of academic interest in the relationship between school "quality" and other 
outcomes such as wages (Card and Krueger, 1992; Betts, 1995), graduation from high school and 
college attendance (Evans and Schwab, 1995). 3 Typically the outcome measure is regressed on a 
host of factors such as individual and family background variables, and measures of school inputs 
such as class size, teacher experience and education, and expenditures per pupil. 4 

In an often cited article, Eric Hanushek states that "differences in [school] quality do not 
seem to reflect variations in expenditures, class sizes, or other commonly measured attributes of 
schools and teachers" (Hanushek, 1986, p. 1 142). To back this claim, he cites numerous studies 
which employ an educational production function methodology and show conflicting results as to 
the importance of schooling resources. He concludes that there is "no strong evidence that 
teacher-student ratios, teacher education, or teacher experience have an expected positive effect on 
student achievement" and that "there appears to be no strong or systematic relationship between 
school expenditures and student performance" (Hanushek, 1986, p. 1 162). These mixed findings 
combined with the more robust result that individual and family background characteristics can 
explain the majority of variation in student test scores, has led some (but not necessarily Hanushek) 
to jump to the conclusion that schools and teachers may not matter, at least in the sense that 
additional monies spent on educational resources are wasted. 

How can these results be explained? If we accept Hanushek's interpretation of the literature 



The notion that there is an estimable education production function for a set of 
individuals within or across classes or teachers or schools or school districts is not unchallenged 
(Monk, 1992). Like any model, the education production function is certainly a simplification of 
reality, but it is a useful tool. This is particularly true for policy purposes because most 
applications focus on manipulate, measurable inputs rather than on intangible variables or 
amorphous constructs. 

The relationship between wages and test scores has recently been examined by Murnane, 
Willett, and Levy (1995). 

4 The focus of our discussion is on schooling inputs. It is quite likely, however, that there 
are unobservable student characteristics such as motivation which are typically omitted from 
educational production functions. 
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and methodology underlying the studies he reviewed, one possibility is that (public) schools have a 
suboptimal allocation of resources (allocative inefficiency) and/or do not operate on the production 
possibility frontier (technical inefficiency). In these cases additional teacher inputs or smaller class 
sizes would not necessarily imply higher output, ceteris paribus. 5 This result does not imply that 
schooling resources may never affect student achievement positively, simply that given the way 
public schools are organized additional resources do not make much systematic difference. This may 
also be due to lack of variation across school settings in variables like class size, making it difficult 
to identify the effects of such characteristics. 

On the other hand, it may be premature to reach strong conclusions about previous research. 
It is possible to challenge Hanushek's "consensus" view of the literature. He reached his conclusions 
by noting the direction of estimated input effects on student achievement, along with whether they 
were statistically significant, and simply tallying ("vote counting") the number of statistically 
significant positive and negative coefficients. A recent "meta-analysis" by Hedges, Laine and 
Greenwald (1994) using the same set of studies reviewed by Hanushek reached a very different 
conclusion. Their basic argument was that the pattern of estimated coefficients is these studies 
suggested there were indeed systematic positive effects. 6 However, Hanushek (1994) in a reply to 
Hedges et al., effectively rejected most of their arguments. 

A more compelling argument against accepting Hanushek's conclusion is that many of the 
studies he reviewed are "bad studies" and should not be given much weight; Hanushek made little 
attempt to separate out studies on the basis of quality save that they had been published. There are 
good reasons to believe that many educational production function studies, particularly those 
completed in the 1970s, had major deficiencies in empirical methodology and available data. If the 
methodological objections are reasonable then an indiscriminate review of the literature will not yield 



5 There is some evidence that public schools and school districts are in fact inefficient 
with regard to resource allocation (Callan and Santerre, 1990) and production technology 
(Grosskopfet al., 1995). 

6 A follow-up study using updated research reaches the same conclusion (Laine et al 

1995). 
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an accurate assessment of the effects of schooling inputs on student outputs. 7 

Data deficiencies imply several problems. The most important of these is that key variables 
may have been omitted from estimated test score models, potentially leading to biased estimated 
coefficients of the included variables. For example, many early studies were unable to control for 
prior achievement using a pre-test score to net out individual ability, as is now generally accepted 
to be important (Boardman and Mumane, 1979; Hanushek, 1979; Hedges et al., 1994). Variables 
representing school and teacher "quality" that are used in most production function studies are 
typically very crude. For example, teacher degree levels and years of experience may be only 
weakly related to teaching skill. Degree level alone does not distinguish between colleges of 
differing quality, nor when the degree was granted, nor does it convey any information about college 
major, certification requirements fulfilled, or subsequent professional development. Teacher 
motivation, enthusiasm, and skill at presenting class material are likely to influence students' 
achievement, but are difficult traits to accurately measure and are thus omitted from standard 
regression analyses. 

Production function studies which have used more refined measures of teacher inputs have 
found more consistent results. Monk and King (1994) report that teacher subject matter preparation 
in mathematics and science does have some positive inpact on student achievement in those subjects. 
Measures of the selectivity of teachers colleges have also been shown to be positively related to 
student achievement (Ehrenberg and Brewer, 1994). The latter result most likely reflects the fact 
that the selectivity measure captures teacher ability. The few studies which have had measures of 
teacher (verbal) ability, for example in the form of a test score, have found a much more robust 
positive relationship to student achievement (Coleman et al., 1966; Ehrenberg and Brewer, 1995; 



7 With a few exceptions most educational production function studies use simple linear 
ordinary least squares (OLS) regression. However, schooling inputs may not be exogenous as 
OLS requires. For example, highly motivated teachers likely locate in a nonrandom manner 
across school districts, schools within a district, and classes within a school. Parents similarly 
self-select their children into school districts which most closely match their preferences, 
including willingness to pay for schooling (Goldhaber, 1995). There have been few attempts to 
control for this potential endogeneity (Ehrenberg and Brewer 1994, 1995; Akerhielm 1995) 
largely because it is extremely difficult to find suitable instruments for schooling inputs, so it is 
difficult to assess its importance. 
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Ferguson, 1991) than those using other teacher characteristics. 

A further issue with regard to teaching "inputs" is that what actually takes place in the 
classroom (the process of teaching and learning) may not be captured by teacher characteristics alone 
(Mumane and Phillips, 1981). This includes both the specifics of the curriculum, the way in which 
material is conveyed to students by teachers, and the dynamics of student and teacher interaction in 
any classroom There is a long tradition among education researchers investigating the relationship 
between teacher behavior and student outcomes (see Brophy and Good, 1986, for a review). 
Although this "process-product" research often uses a univariate approach (e.g., simple pairwise 
correlations) and small samples, it has produced some relatively strong results, particularly in regard 
to the importance of curricula exposure and some pedagogical strategies. If such variables are 
important predictors of student achievement and are not captured by included school and teacher 
characteristics, traditional educational production functions may again yield biased results. 

Data deficiencies may also have led to significant measurement error problems in previous 
studies. For instance, it is possible that schooling inputs, particularly class size and expenditures per 
pupil, are measured with error. Some error likely arises from aggregation of variables to school 
level. 8 Rather than class size, studies often utilize total school enrollment divided by total number of 
teachers as an average pupil-teacher ratio. This can lead to dramatically different estimates of the 
effects of school resources on achievement (see, for instance, Akerhielm, 1995). Of particular 
concern is that "teacher characteristics" used in many studies are not in fact those of any individual 
teacher but rather a school level variable such as mean years of experience, percentage of teachers 
with at least a Masters degree, or mean teacher salary. Use of these measures ignores the 
considerable variation in teacher characteristics that may exist within a school, and makes it 
problematic to accurately access the impact of specific teacher characteristics on achievement. 

The conventional view that observable school inputs, and teachers in particular, do not 
positively impact student achievement rests on somewhat shaky empirical grounds. The main 
problem is likely to be omitted variable bias arising from inadequate data and extremely crude proxies 
for teacher skill found in most educational production functions. In the absence of more refined 

8 For a discussion of aggregation bias in this context see Hanushek, Rivkin, and Taylor 
(forthcoming). 
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measures, it is important to establish whether the omission of unobservables biases the estimates of 
the variables that are included in the educational production functions. If this is the case it calls into 
serious question the usefulness of this approach for public policy. In the next section we lay out the 
problem and strategies for dealing with it. 



III. Econometric Methodolog y 

Traditional economic models of student achievement generally build upon the theoretical 
construct known as the educational production function (Hanushek, 1986). In these models the 
achievement of student i at school j, Y y , is regressed on a vector of individual and family background 
variables (including some measure of prior ability or achievement), X, } , and a vector of schooling 
resources, Sj, which do not vary across students, and a random error term: 



T. = 

V 



PY 



+ Y* - s 



( 1 ) 



For now, assume Sj may consist of school, teacher, or class specific variables, p is the return to 
individual and family background characteristics and y is the return to schooling resources. If (1) 
is correctly specified, Ordinary Least Squares (OLS) estimation will yield consistent estimates of p 
and y . The overall importance of schooling factors Sj can be ascertained by performing an F-test of 
the hypothesis that the coefficients of the schooling variables are jointly equal to zero. 

As discussed in section II there are legitimate reasons for believing that standard educational 
production function models are subject to omitted variables bias. Assume, for example, that the 
vector of schooling resources S (dropping the subscripts) can be decomposed into two parts: 
observable characteristics, Z„ such as class size, teacher experience, teacher degree level, or per pupil 
expenditure, which are included in the model; and unobservable characteristics, Zo, such as teacher 
skill, behavior and motivation, and classroom peer effects, which are omitted from the model: 

S j = [z v z 2,l ( 2 ) 

Hence the true model is: 

Y a = P X ij + Y,Z U + + S (3) 
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but we estimate: 



where the error term now consists of unobservable schooling characteristics as well as a random 
component (where v ij =Y 2 Z 2 j+e i j). 

Thus the omission of 7^ may cause two problems. First, the total effect of schooling resources 
on student outcomes could be understated because omitted factors (Z?) will not be included in the 
explained portion of the variation in student achievement. Second, (4) may yield biased estimates of 
the effects of particular schooling resources on student outcomes, as shown in (5): 



P* = P + y 2 ( z ' z 2 )'\ z 2 ' x ) 
V = y, ♦ Y 2 (z/z/Vz/z,) 



The obvious solution to this problem is to utilize data which includes the relevant variables in Zj, or 
at least good proxies for them. As noted in the discussion in section II, this might be done by adding 
measures of teacher behavior or teacher ability, or refined measures of peer effects. While recently 
available educational data have made this possible, it is unlikely that any dataset will contain sufficient 
information to adequately capture Zj. 

In the case in which individual and family background variables and included schooling 
resources are uncorrelated with excluded schooling resources ECX'Zj) = ECZ.'Zj) = 0, OLS estimation 
of (4) will yield consistent and unbiased estimates of p and y However, OLS gives estimates of the 
variance of P and y , which are biased upwards, thus preventing valid inferences about the estimated 
coefficients of either individual and family background variables or schooling variables. 

Maintaining the assumption that Z, is uncorrelated with included regressors, one possible 
solution is to treat 7^ as a random disturbance specific to a school. For example, suppose 7^ 
contains school level predictors which do not vary across students, and can be reasonably assumed 

to be random across schools. In this case, denote the vector Z, by the random variable Zj . The 
model is then: 
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( 6 ) 



y <, = P*# + r, z » + *i ' «, 

This is a standard random effects model in which there are N groups (say schools) with T student 
observations per school (NT total observations). This model can be estimated by Generalized Least 
Squares (GLS) to yield consistent, efficient and unbiased estimates of the effect of observed schooling 
resources on student achievement Z,. This model has the well known advantage that the desirable 
properties of the estimator does not rely on the number of student observations T being infinite. 
However, it is predicated on the orthogonality of the random effect with the included regressors, as 
well as requiring a distributional assumption about z y 

Note that a version of this random effects model is possible in which observed schooling 
characteristics are omitted. In other words, treat the total effect of schools as constant across 
students but random across schools. Denote this random effect aj. 



Y, = $X.. ♦ a, «■ «= 



(7) 



In this model the schooling resource variables are assumed uncorrelated only with individual and 
family background variables, whereas in (6) unobservable schooling variables (treated as a random 
effect (Zj)) are assumed uncorrelated with both individual and family background variables (X) and 
observable schooling characteristics (Z,). By explicitly incorporating the school specific variables 
which do not vary across students (Z,) in (6) we eliminate or reduce the correlation between X and 
a,; of course, if the omitted effect remains correlated with X or Z„ the GLS random effects estimator 
is biased (Hsiao, 1986, p.52). 

The assumption that either E(X' Zj) = 0 or E(Z,'Z 2 ) = 0 is a strong one; if it is violated 
standard OLS production functions will yield biased estimates of the effects of schooling resources 
on student outcomes. Similarly if a school specific random error component (Zj or aj) is included in 
a production function but is correlated with included regressors (X or both X and Z,) GLS will also 
yield biased and inconsistent estimates of the effects of schooling resources on student outcomes. 
For example, in (4), the direction of the bias will depend on the signs of y 2 (the true effects of 
schooling unobservables), EfX'Zj) and EfZ/Zj). While it might be hypothesized that y 2 > 0, the 
relationship between omitted schooling variables and the included regressors is an empirical issue. 
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Suppose that children from upper income families attend schools with more motivated teachers but 
that teacher motivation is unobservable, i.e., EfX'Zj) > 0. In this case, assuming teacher motivation 
positively impacts student achievement, the estimation of the educational returns to increases in family 
income will be overstated. Similarly suppose that younger, more inexperienced teachers are the most 
motivated such that EfZ/Z^) < 0. In this case the estimated effects of teacher experience on student 
achievement would be understated in (4). 

If panel data are available, the standard technique to account for omitted variables bias is to 
estimate a fixed effects model. In this case let ctj be a schooling specific effect which does not vary 
across individuals but is specific to each school. In a sample of i =1...N schools with T observations 
per school, one can then estimate 

y o = P x o + a j * e o ( 8 ) 

using ordinary least squares to obtain unbiased estimates of the effect of individual and family 
background variables (X) and the total effect of schooling resources (S) on student achievement. 
This model does not require the restrictive assumption that the unobservable schooling variables are 
uncorrelated with included variables since the effect of all school specific unobservables is captured 
by o^. The estimates of p are also consistent. However, while the estimates of a jt the schooling 
effects, are unbiased, their consistency relies on the number of observations being large (infinite). 
They also may not be fully efficient. 

Determining whether a random effects model specification ((6) or (7)) or a fixed effects model 
(8) is the "correct" model depends partly on the relationship between the omitted effects and included 
regressors. If schooling effects are correlated with individual and family background variables then 
(7) is not appropriate. Similarly, (6) is not appropriate if observable schooling variables are 
correlated with unobservable schooling effects. Whether a fixed or random effects model is 
appropriate can easily be tested using a Hausman test. The null hypothesis is that the coefficient 
estimates from the fixed effects model (8) do not differ systematically from the estimates of the 
random effects specification of the model (7). 
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The difficulty with the fixed effects specification for our purposes is that while the tottil effect 
of schooling resources on achievement can be quantified, the effects of particular observable 
resources (Z,) which are of interest for policy purposes cannot be ascertained since inclusion of Z, 
along with ctj would lead to perfect collinearity. 9 One possible solution is to estimate an auxiliary 
regression in which the estimates of are regressed on observable schooling variables Z, to obtain 
consistent estimates of y, (see, for example, Rauch, 1993, for an application of this methodology). 10 
Since a j is an estimate derived from (8), (9) must be a estimated by GLS to ensure the coefficient 
standard errors are unbiased. 



a j = 



Y 



IV 



(9) 



IV. Data 

The data used here are derived from the first two waves of the National Educational 
Longitudinal Study of 1988 (NELS). This is a nationally representative survey of about 24,000 8th 
grade students conducted in the spring of 1988. 18,000 of these students were resurveyed in 10th 
grade (spring 1990). Students provided comprehensive information on themselves and their families 
(their race/ethnicity, sex, family structure), supplemented by a parental survey in 1988 (providing 
information on, for example, family income). At the time of each survey students took one or more 
subject based tests in mathematics, science, writing, and history. The data therefore permits the 
estimation of "value-added" or gain score production functions which control for previous knowledge 
or ability. In this paper, we confine our attention to those students who took the mathematics 



One solution to this problem is an instrumental variables estimator developed by 
Hausman and Taylor (1981) which permits the efficient estimation of y,. They suggested 
estimating y , using elements of X uncorrelated with a- as instruments for Z,. A necessary 
condition to implement this method is that the number of elements of X uncorrelated with cl must 
be greater than the number of elements of Z, that are correlated with a y 

10 This method is a "weak" version of a the currently popular "hierarchical linear models" 
(HLMs) technique widely used within education (although rarely used by economists). HLMs 
estimate "within school" (typically slope) parameters which are used as outcomes in school level 
between school" equations. See, for example, Bryk and Raudenbush (1988). 
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achievement test in the 8th and 10th grade, the 10th grade score being modelled as a function of the 
8th grade score and other variables. 11 The tests were carefully designed by the Educational Testing 
Service to assess mathematics knowledge. While all students took the same test in the 8th grade, 
tests of varying difficulty were given to students in the 10th grade depending on their 8th grade score 
in order to guard against "ceiling" and "floor" effects. The tests were given a common scale using 
Item Response Theory (IRT). 12 

The unique feature of NELS is that it provides detailed teacher and class level information that 
is tied directly to individual students. In other words, the characteristics of each 10th grade 
mathematics teacher (sex, race/ethnicity, degree level, experience, certification, etc.) who taught 
students taking the 10th grade mathematics test is known. In addition, some characteristics of the 
classes which that teacher taught are also available (e.g. class size, track 13 , class composition). This 
feature of the data makes NELS one of the most comprehensive and well designed studies available 
for conducting educational production function analyses. School administrators provided information 
on their schools: for example, in regard to the composition of the teaching staff (percentage with a 
Masters degree, percentage white, etc.) and school quality indicators (e.g. percentage of the school's 
graduates going on to four year colleges). 

We confine our attention to a sample of 5,149 10th grade public school students. Public 
school students are examined because of the difficulties of controlling for the non-random assignment 
of students to public and private schools (Goldhaber, 1996). Further, any policy conclusions 
resulting from our analyses apply primarily to public schools. These students are drawn from 638 
schools, with 2,245 individual mathematics teachers. The sampling frame of NELS is such that there 
are not only multiple students per school and per teacher, but there are also multiple classes per 

“ The 10th grade test was administered during the spring of 10th grade. Unfortunately 
no data are available on students' 9th grade experiences. 

12 IRT is a method that uses the pattern of right, wrong, and omitted responses to the 
questions actually administered on each test .and the difficulty, discriminating ability, and 

guessability of each question to place each student on a continuous scale regardless of the test 
he or she was given (see Rock and Pollock, 1991, for a discussion of IRT and the NELS data). 

13 A recent study of the effects of tracking on student achievement using the NELS data is 
Argys, Rees, and Brewer (1996). 
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teacher; there are 3,498 classes in our sample. This is also a unique advantage of the NELS data 
which, at least in principle, permits the estimation of a broader class of models than is usually the 
case. Means of all variables used in our analyses are found in Appendix Table 1 

V. Results 

Standard OLS Educational Production Functions 

We begin by estimating standard educational production functions by ordinary least squares. 
Throughout our analyses, the dependent variable is the 10th grade mathematics test score. We group 
our explanatory variables into four sets; individual and family background variables (including sex, 
race/ethnicity, parental education, family structure, family income, and 8th grade math test score 14 ), 
school variables (including urbanicity and regional dummies, school size, the percentage of students 
at the school who are white, the percentage of students at the school who are from single parent 
families, the percentage of students who typically go on to four year colleges, and the percentage of 
teachers at the school with at least a Masters degree), teacher characteristics variables (including sex, 
race/ethnicity, years of experience at the secondary level, whether the teacher is certified in 
mathematics, the teacher's degree level and whether his or her BA major or MA major are in math), 
and classroom variables (class size and percentage of minority students). As noted previously, a 
unique feature of our NELS data is that the teacher and classroom variables are specific to the 
students 10th grade mathematics class. 

Table 1 contains the basic model results. We do not show the coefficients of individual and 
family background variables though they are included in each model. These variables alone explain 
approximately three quarters of the variation in the 10th grade math score. In our sample, most of 
the estimated coefficients of these variables are statistically significant in the expected direction. For 
example, years of parental education and family income are positively related to test scores, a result 
typical of the literature. Black and Hispanic students have lower predicted scores on average, as do 

14 Models were also run with a quadratic form of the base year test score. The estimated 
coefficient of this variable was negative and statistically significant, while the coefficient of the 
base year test score remained positive and significant, suggesting a nonlinear relationship between 
prior and current achievement. However, the inclusion of the quadratic term in the model did not 
change the estimated effects of other variables of interest, and the results are not shown here. 
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those from families with no mother in the household. 

[Table 1 about here] 

Column (1) of Table 1 adds the set of school characteristics to the model. Only two of the ten 
variables are statistically significant, though most coefficients are the expected sign. However, an F 
test of the hypothesis that the coefficients of the schooling variables are jointly equal to zero is 
rejected at the 1 % level. Column (2) shows the results when teacher characteristics are added to the 
model. Here eight of the eleven teacher variables have statistically significant coefficients. An F test 
of the hypothesis that the coefficients of the teacher characteristics are jointly equal to zero is easily 
rejected at the 1 % level. We find that students of more experienced teachers have higher scores. 
Female teachers are associated with higher test scores and black teachers with lower test scores; the 
relationship between teacher race, ethnicity and gender and student scores is likely more complex 
than this simple linear model allows but we do not explore the issue in here (for a more detailed 
analysis of this issue Ehrenberg, Goldhaber and Brewer, 1995). 

Column (3) of Table 1 adds two classroom level variables to the production function: class 
size which is positively associated with achievement, and the percentage of minority students in the 
class which is negatively associated with achievement. The class size result is counterintuitive 
(though the magnitude of the estimated coefficient is small) but not atypical of production function 
results. As noted earlier, Akerhielm (1995) found a similar result with NELS data which she 
attributed to the nonrandom assignment of students to classes. Again an F test comparing the models 
in (2) and (3) easily rejects the hypothesis that the coefficients of these classroom variables are jointly 
equal to zero. 

Before adding teacher behavior variables it is worth considering more closely the teacher 
characteristic results which contrast with many earlier studies which have found little relationship 
between such variables and student achievement. As shown in columns (2) and (3) of Table 1, the 
estimated coefficients of whether the teacher has a Bachelors degree and whether she has a Masters 
degree are negative (and in the case of BA, statistically significant) relative to the omitted category 
of no degree. This implies that teachers with Bachelors and Masters degrees are less effective than 
those without any degree, clearly a counterintuitive finding. The results for teacher certification are 
similar in that we find the coefficient on teacher certification to be statistically significant and 
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negative. However, these models also include dummy variables indicating whether the teacher is 
certified in math, has a Bachelors in math, and has a Masters in math. These variables allow us to 
distinguish between teachers who are teaching mathematics classes and have a major in math (BA or 
MA), teaching mathematics classes and are certified in math, and those who are teaching mathematics 
but do not have specific math training. For example, the total effect of a teacher having a Bachelors 
degree in math is the sum of the coefficients of BA and BA major in math, whereas the effect of 
teacher who has a non-math BA is simply the coefficient of the BA variable. 

Traditional education production functions do not include subject specific teacher degree and 
certification information. However, at least in our sample, the use of this more detailed data is critical 
in interpreting the effects of these teacher characteristics on student achievement. This can be seen 
in Table 2. Column (1) of Table 2 shows a model in which math specific degree and certification 
information is omitted. The results here would lead one to one to the conclusion that teacher degrees 
and certification have no impact on student achievement. This is in line with much of the previous 
literature. However, the results reported in column (3) of Table 2 (reproduced from column (3) of 
Table 1) would lead to a quite different conclusion. Here we show that a teacher with a BA in math 
or an MA in math has a statistically significant positive impact on students' achievement, while a 
teacher with a non-math BA or an MA has a negative impact on students' math achievement (all 
relative to the omitted category of no degree). We find similar results with teacher certification, when 
we estimate models with (columns (1) and (3) of Table 2) and without (columns (2) and (4) of Table 
2) a set of state dummy variables indicating the location of the high school. We estimate models with 
state dummies to control for the differing certification requirements across states. 

[Table 2 about here] 



Adding "Teacher Behavior" Variables 

As noted in the previous section, many education researchers have suggested that teacher 
characteristics do not begin to capture the ways in which teachers affect student achievement. In 
particular, teacher behavior is typically not included in educational production function analyses 
which treats the process of schooling as a "black box". Few attempts have been made to ut iliz e such 
concepts in multivariate analysis, partly because measuring them is extremely difficult (Brewer and 
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Stasz, 1996), and partly because the choice of variables for inclusion is somewhat arbitrary. One 
useful effort to do this with data on elementary schools is Mumane and Phillips (1981). They include, 
in production functions explaining vocabulary test scores, variables such as the percentage of time 
the teacher uses subgroups, demonstrations, individualized work, and whether the teacher views their 
job as explaining subject matter. The authors find that these behavior variables explain a larger 
proportion of test score variance than teacher characteristics alone, although a model including both 
characteristics and behavior measures best fits the data. 

Similar information on teacher behavior is included in NELS. This includes the percentage 
of the teacher's time in class devoted to small groups and individualized instruction, the percentage 
of time maintaining order and doing administrative tasks, whether the teacher uses oral questions 
frequently and emphasizes problem solving, whether the teacher has no control over curriculum 
content, teaching technique and disciplinary policy, and whether the teacher feels well prepared. 15 
We selected these items partly to replicate the Murnane and Phillips analysis at the secondary level 
and partly to capture newer conceptions of mathematics teaching (small groups, problem solving) 
as reflected in the National Council of Teachers of Mathematics (NCTM, 1991) standards. 

Column (4) of Table 1 shows a model including teacher behavior but not teacher 
characteristics; column (5) includes both sets of variables. The model with teacher behavior variables 
is marginally better at explaining test scores than the characteristics model, but including both sets 
of variables together provides for the best fit of the data, as was the case in Murnane and Phillips 
(this is confirmed by F-tests of the hypotheses that the added coefficients are jointly equal to zero). 
However, the estimated coefficients of the teacher characteristics variables do not change much when 
the behavior variables are added; the reverse is also true. This suggests that there my be no strong 
relationship between our set of teacher characteristics and any teacher behavior variable. We 



15 These items are based on teacher survey responses. Burstein et al. (1995) assess the 
validity of NELS instructional strategy items using an enhanced version of the teacher survey 
instruments, teacher interviews, and collection of class assignments and homework, with a small 
sample of teachers in the original NELS sample. Their results suggest such survey items and 
response scales need to be treated cautiously. 
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confirmed this in auxiliary regressions of teacher behavior on teacher characteristics. 16 

Several results are worth highlighting. Students with teachers who have little or no control 
over their teaching technique have significantly lower test scores. Variables associated with NCTM 
type standards, such as teaching in smaller groups and emphasizing problem solving, appear to lower 
student math scores. This does not necessarily imply that these teaching techniques are invalid but 
may simply indicate that these methods do not improve achievement on traditional standardized 
tests. Murnane and Phillips (1981) found a similar statistically significant negative coefficient on 
percentage of time devoted to subgroup instruction for sixth grade vocabulary. In fact, all of our 
results, at the secondary level, are strikingly consistent with their results at the primary level. On net, 
it appears that these types of variables should be included in educational production functions but 
since our focus here is primarily on biases in traditional education production functions, we do not 
discuss these variables and do not include them in subsequent models shown. Our conclusions below 
are unaffected by this simplification. 

School, Teacher, and Class One-Way Random and Fixed Effects Models 

Table 3 shows the results of one-way random effects models with included school (column 
(f ))> teacher (column (2)), and class (column (3)) random effects, as shown in equation (6). These 
models were also estimated omitting observed variables as in equation (7). The results are very 
similar to those of the base model shown in Table 1. However, a Lagrange multiplier test confirms 
that the random effects specification of the model is superior to a standard OLS specification. 
Following (8), we also estimate 1-way fixed effects models which allow us to test for unobservable 
school, teacher, and class effects. 17 Hausman tests suggest that random effects specifications are 



We ran a series of simple regression models in which each teacher behavior variable 
was regressed against observable teacher characteristics variables. There were few consistent 
results across models. Further, the fit of these models was extremely poor: the R-squared was 
typically less than 5%. 

There have been several previous studies which have estimated teacher fixed effects and 
found them to be similarly important (see Hanushek, 1986, p. 1159). 
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appropriate for these data relative to fixed effects models. 18 

[Table 3 about here] 

Column (1) of Table 4 shows the results of a school level fixed effects model, column (2) 
shows the results of a teacher level fixed model, and column (3) shows the results of a class level 
fixed effects model. In these fixed effects models we cannot include observed variables at the same 
level as the fixed effect (e.g. teacher observed variables with a teacher specific effect), nor can we 
include higher level observed variables than the level of the fixed effect (e.g. school level observed 
variables with a class level effect). A comparison of the estimated teacher and class coefficients in 
our base model that includes observed school, teacher, and class variables (column (6) of Table 1) 
with our model that replaces the observed school variables with a school specific effect (column (1) 
of Table 4) shows few changes in the estimates of teacher and class variables when a school level 
fixed effect is included in the model. 19 

[Table 4 about here] 

The explained portion of the variation of student achievement when we move from a model 
with our complete set of observed characteristics to our model with class effects, rises from .77 to 
.93. Adding a teacher fixed effect to the standard model also explains a significantly higher portion 
of the variance in test scores than a model with teacher observable characteristics, a result consistent 
with the findings of other studies which have estimated teacher fixed effects models. 20 To determine 
whether these models better fit the data, we perform F-tests of the hypotheses that the coefficients 
of the fixed effects are jointly equal to zero. In all cases we are able to reject these hypotheses at the 
1 percent significance level. Thus, unobservable school, teacher, and class effects do play a 



18 The student data used by Montmarquette and Mahseredjian (1989) suggests that a two 
way (school and class) random effects, rather than a fixed effects, education production function 
model is appropriate. 

19 However, when teacher behavior variables are included in the school fixed effects 
model, many of the teacher behavior variables which were significant in the OLS specification of 
the model shown in Table 1 become statistically insignificant. This may suggest that teacher 
behavior is strongly associated with unobserved aspects of school organization, and we control 
for this with the inclusion of the school level effect. 

See Hanushek, 1986, p. 1 159 for citations and discussion of these studies. 



significant role in determining student outcomes but these effects do not appear to be jointly 
correlated with included explanatory variables. 

Auxiliary Models Using Estimated Fixed Effects 

To obtain consistent estimates of individual teacher observable variables we regress our 
estimates of teacher fixed effects on the set of observed teacher characteristics. 21 As we found earlier, 
the auxiliary regression results suggest that teacher certification, experience, and degrees in math are 
all statistically significant in the expected direction. Further, the magnitudes and significance of these 
coefficients are larger in each case. The results are shown in table 5. 

[Table 5 about here] 



These results confirm our earlier findings regarding the importance of teacher characteristics. The 
marked difference in the coefficient estimates suggests that although the set of teacher variables are 
not jointly correlated with a teacher specific effect, these individual variables may be. 

VI. Conclusion 

In this paper we assess the importance of educational resources in explaining student 
achievement on a tenth grade mathematics standardized test. Using data from the National 
Longitudinal Study of 1988 which permit us to link students with a particular teacher and math class, 
we find that traditional (OLS) educational production function models do show some educational 
resources to be significant in influencing tenth grade mathematics test scores. Although school level 
variables do not, in general, seem to have much affect on student achievement, some teacher 
characteristics do. Teachers who are certified in mathematics, and those with Bachelors or Masters 
degrees in math, are identified with higher test scores. Auxiliary regression models in which an 
estimated teacher fixed effect is regressed on observable teacher characteristics to obtain consistent 
estimates of these effects, strengthens our conclusion that these variables influence student 



21 This methodology may also be applied to school and class effects. We choose to focus 
here on teacher effects as a confirmation of our earlier positive results with respect to key teacher 
characteristics. 
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achievement. Extending the work of Murnane and Phillips (1981) we also find that some aspects of 
teacher behavior influence student achievement, independent of teacher characteristics. 

Our findings regarding teacher traits may be due to the unique nature of the NELS data 
among national surveys in providing a direct student-teacher-class link. This link enables us to avoid 
problems with aggregation that may have plagued earlier studies. Also, subtle differences in model 
specification can result in very different interpretation of whether teachers, for example, affect student 
outcomes. Student achievement models estimated with variables specifying whether a teacher has 
a BA or an MA but without the subject of the degrees, show teacher degree to be statistically 
insignificant; this is also true of teacher certification. When degree subject is added, teachers with 
mathematics degrees are clearly associated with higher student scores on the NELS mathematics test. 

The NELS data permit us to estimate a variety of econometric models including one-way 
fixed and random effects models. Our results suggest that while observable school, teacher or class 
variables account for a relatively small fraction of the variation in student test scores, unobservable 
factors associated with these schooling characteristics are important. However, we find that 
unobservable school, teacher, and class effects do not appear to be correlated with observable 
variables included in our models, and thus do not cause biased estimates of the effects of these 
variables. 
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Table 1 

Educational Production Functions with Individual, School, Teacher, 
and Class Variables (absolute value t-statistics) 

Standard OLS Models 
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Table 2 

Comparison of Selected Coefficients from Educational Production Functions 

(absolute value t-statistics) 





(1) 


(2) 


(3) 


(4) 


Teacher years experience at secondary level 


0.022 (1.9) 


0.023 (2.0) 


0.016(1.4) 


0.017(1.5) 


Teacher is certified 


-0.072 (0.1) 


-0.305 (0.5) 


-2.265 (2.3) 


-2.412 (2.4) 


Teacher certified in math 


— 


— 


2.150 (2.2) 


2.024(2.1) 


Teacher has B.A. 


-0.183 (0.7) 


-0.047 (0.2) 


-0.736 (2.4) 


-0.578 (1.8) 


Teacher has B.A. major in math 


~ 


— 


0.823 (3.5) 


0.748 (3.1) 


Teacher has M.A. or more 


0.130 (0.5) 


0.162 (0.7) 


-0.055 (0.2) 


-0.010 (0.0) 


Teacher has M.A. major in math 


- 


— 


0.606 (2.2) 


0.536 (1.9) 


State Dummies 


No 


Yes 


No | 


Yes 


Adjusted 


.767 


.770 


.768 


.772 



All models contain individual and family background variables, school variables, teacher 
characteristic variables (except as specified in table), and class variables. Table 1, col. (3), 
reproduced in Table 2, col. (3). 





Table 3 

Educational Production Functions with Individual, School, Teacher, and Class Variables 

(absolute value t-statistics) 

One Way Random Effects Models 



Category 


Dependent Variable: 10th Grade Test Score 


(1) I (2) (3) 


School Variables 








Urban 


-0.123 (0.4) 


-0.148 (0.5) 


-0.142 (.55) 


Rural 


-0.332 (1.2) 


-0.407 (1.6) 


-0.323 (1.3) 


Northeast 


0.396 (1.1) 


0.428(1.3) 


0.426(1.3) 


North central 


-0.238 (0.8) 


-0.165(0.6) 


-0.148 (0.0) 


West 


0.081 (0.2) 


0.149 (0.5) 


0.012 (0.0) 


School size* 


0.078 (0.4) 


0.087 (0.4) 


0.167(0.8) 


Percent 10th graders white 


-0.031(5.1) 


-0.030 (5.0) 


-0.029 (4.9) 


Percent school graduates who attend four-year college 


0.019 (2.8) 


0.019(2.9) 


0.022 (3.4) 


Percent teachers with M.A. at school * 


-0.002 (0.3) 


-0.002 (0.4) 


.001 (0.1) 


Percent school from single-parent families 


-0.006 (0.8) 


-0.006 (0.9) 


-.005 (0.7) 


Teacher Characteristic Variables 








Female teacher 


0.546 (2.7) 


0.598 (2.8) 


0.572 (2.8) 


Black teacher 


-1.019(1.5) 


-0.982 (1.8) 


-1.123 (2.1) 


Hispanic teacher 


1.086 (1.5) 


1.213 (1.7) 


0.991 (1.4) 


Asian teacher 


0.948(1.1) 


0.896 (1.0) 


0.017 (0.7) 


Teacher years experience at secondary level 


0.019 (1.6) 


0.012(1.4) [ 


0.021(1.7) 


Teacher is certified 


-2.313 (2.3) 


-2.345 (2.3) 


-2.783 (2.1) 


Teacher certified in math 


2.185 (2.2) 


2.230 (2.2) 


2.445 (2.4) 


Teacher has B.A. 


-0.726 (2.3) 


-0.666 (2.0) 


-0.663 (2.0) 


Teacher has B.A. major in math 


0.847 (3.5) 


0.847 (3.4) 


0.781(3.2) 


Teacher has M.A. or more 


-0.092 (0.4) 


0.570 (0.2) 


0.160 (0.6) 


Teacher has M.A. major in math 


0.621 (2.1) 


0.567(1.9) | 


0.520 (1.8) 


Class Variables 








Student class size 


0.046 (3.1) 


0.046 (3.0) 


0.056 (3.7) 


Percent minority students in class 


-0.043 (6.7) 


-0.041 (6.5) 


-0.042 (6.5) 


School Random Effects 


Yes 


No 


No 


Teacher Random Effects 


No 


Yes 


No 


Class Random Effects 


No 


No 


Yes 


Adjusted 

n J 1 1 AAA 


.770 .770 .770 



^Coefficient multiplied by 1000. 
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Table 4 

Educational Production Functions with Individual, School, Teacher, and Class Variables 

(absolute value t-statistics) 

One Way Fixed Effects Models (OLS) 



Category 


Dependent Variable: 10th Grade Test Score 


(1) 


i (2) 


I (3) 


Teacher Characteristic Variables 








Female teacher 


0.674 (3.5) 


— 





Black teacher 


-0.809 (1.6) 


— 


— 


Hispanic teacher 


1.360 (1.9) 


— 


— 


Asian teacher 


0.886 (1.0) 


— 


— 


Teacher years experience at secondary level 


0.016 (1.4) 


— 





Teacher is certified 


-2.617 (2.6) 


— 





Teacher certified in math 


2.240 (2.3) 


-- 





Teacher has B.A. 


-0.705 (2.3) 


— 





Teacher has B.A. major in math 


0.763 (3.3) 


— 





Teacher has M.A. or more 


0.503 (0.2) 


_ 





Teacher has M.A. major in math 


0.609 (2.2) 


— 


— 


Class Variables 








Student class size 


| 0.040 (2.9) 


0.054 (1.7) 





Percent minority students in class 


-0.016 (3.7) 


-0.078 (4.6) 


— 


Individual and Family Background Variables 


Yes 


Yes 


Yes 


School Fixed Effects 


Yes 


No 


No 


Teacher Fixed Effects 


No 


Yes 


No 


Class Fixed Effects || 


No ! 


No | 


Yes 


Adjusted 


.784 


.801 


.815 


* i 


.812 


.888 


.933 





Table 5 

Comparison of Selected Coefficients from Educational Production Functions 

(absolute value t-statistics) 





(i) 

OLS 


(2) 

Random Effects 


(3) 

Auxiliary 


Teacher years experience at secondary level 


0.019 (1.6) 


0.012 (1.4) 


0.030 (2.3) 


Teacher is certified 


-2.499 (2.5) 


-2.345 (2.3) 


-3.552 (3.2) 


Teacher certified in math 


2.271 (2.4) 


2.230 (2.2) 


3.254 (3.0) 


Teacher has B.A. 


-0.797 (2.6) 


-0.666 (2.0) 


-0.935 (2.7) 


Teacher has B.A. major in math 


0.867 (3.7) 


0.847 (3.4) 


1.098 (4.2) 


Teacher has M.A. or more 


-0.134 (0.5) 


0.570 (0.2) 


0.090 (0.3) 


Teacher has M.A. major in math 


0.632 (2.2) 


0.567 (1.9) 


0.885 (2.8) 



The OLS coefficients reported are from Table 1, col. (3). The random effects coefficients reported are 
from Table 3, col. (2). Col. (3) is estimated by GLS. 
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Appendix Table 1 

Variable Means (Standard Deviations) 



| Category 


Mean (Standard Deviation) || 


Individual and Family Background Variables 




| 10th grade mathematics test score 


43.94 


(13.6) 


1 8th grade mathematics test score 


36.57 


(11.7) 


| Female 


0.51 


(0.5) 


Black 


0.09 


(0.3) 


1 Hispanic 


0.11 


(0.3) 


| Asian 


0.05 


(0.2) 


| Parental education 


14.07 


(2.3) 


| Family income 


37152.48 


(31418.82) 


| Mother only 


0.07 


(0.3) 


| Father only 


0.14 


(0.1) 


| School Variables 




1 Urban 


0.20 


(0.4) 


I Rural 


0.41 


(0.5) 


| Northeast 


0.15 


(0.4) I 


1 North central 


0.33 


(0.5) j 


1 West 


0.17 


(0.4) 


1 School size 


1188.29 


(689.1) 


| Percent 10th graders white 


76.44 


(29.1) 


| Percent school graduates who attend four-year college 


39.92 


(18.6) 


Percent teachers with M.A. at school 


50.24 


(22.5) I 


Percent school from single-parent families 


28.79 


(16.2) 


1 Teacher Characteristic Variables 




| Female teacher 


0.46 


(0.5) 


| Black teacher 


0.04 


(0.2) 


| Hispanic teacher 


r o.o 2 


(0.1) 


| Asian teacher 


0.01 


(0.1) 


Teacher years experience at secondary level 


15.50 


(9.0) 


| Teacher is certified 


0.96 


(0.2) 


1 Teacher certified in math 


0.94 


(0.2) 


j Teacher has BA. 


0.77 


(0.4) 


1 Teacher has B.A. major in math 


0.52 


(0.4) 


Teacher has M.A. or more 


0.50 


(0.5) 


| Teacher has M.A. major in math 


0.17 


(0.4) 


| Teacher Behavior Variables 




| Teacher feels very well prepared 


0.72 


(0.4) 


I Teacher has no control over discipline 


0.04 


(0.2) 


I Teacher has no control over technique 


0.01 


(0.1) 


Teacher has no control over context 


0.19 


(0.4) 


Percent teacher time instructing small groups 


11.23 


(12.6) 


Percent teacher time instructing individuals 


18.00 


(16.4) 


1 Percent teacher time maintaining order 


8.39 


(14.9) 


1 Percent teacher time doing administrative tasks 


5.52 


(6.4) 


! Teacher uses oral questions frequently 


0.75 


(0.4) 


Teacher emphasizes problem solving 


0.24 


(0.4) 


Class Variables 




Student class size 


23.37 


(7.0) 


Percent minority students in class 


20.38 


(28.4) 


Sample Size 


5149 H 
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